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ABSTRACT 

Measuring  personality  is  a  complex  task.  Besides  difficult  content-based  problems  there  are  some 
psychometric  issues  to  be  met,  i.  e.  socially  desirable  answering  tendencies.  The  present  study  uses 
simulation-based  objective  personality  measurement  to  meet  these  psychometric  issues.  Computer-based 
“guard”  and  “stray”  scenars  are  developed  and  tested  in  two  samples  of  Special  Forces  applicants 
respectively  soldiers.  A  construct  model  has  been  predefined  and  used  to  develop  behavioural  markers  in 
the  scenars.  These  markers  are  the  basis  for  the  subsequent  analysis.  Results  show  that  a  substantial 
number  of  behavioural  markers  do  not  show  sufficient  variation  and  therefore  have  to  be  removed  from 
further  analysis.  With  the  remaining  markers  the  construct  model  is  evaluated.  The  predefined  model 
couldn  ’t  be  identified  in  the  data,  but  an  alternative  model  could  be  established  at  least  for  the  “guard” 
scenar.  Further  development  of  the  scenars  is  necessary  in  order  to  meet  the  requirements  of  an  efficient 
and  effective  diagnostic  instrument. 


1.0  INTRODUCTION 

Measurement  of  personality  is  a  really  complex  task.  First,  it  appears  to  be  more  difficult  to  define  exactly 
the  different  personal  abilities,  especially  to  differentiate  between  these  constructs.  Second,  to 
operationalize  the  constructs  is  more  difficult  because  it  means  to  deal  with  subjective  perceptions  of 
social  situations.  Third,  beside  these  more  psychological  aspects  there  are  some  psychometric  issues.  The 
classical  questionnaires  which  measure  personality  are  vulnerable  with  respect  to  answering  tendencies. 
This  is  especially  the  case  in  selection  contexts,  in  which  the  candidates  try  to  make  a  positive  impression 
of  themselves.  Answering  tendencies  in  this  context  mean  socially  desirable  answering,  or  in  a  more 
extreme  wording:  faking  results. 

Despite  these  challenges  it  becomes  more  and  more  necessary  to  deal  with  the  measurement  of 
personality.  In  military  contexts,  the  demands  related  to  personal  abilities  rise.  For  the  German  Federal 
Armed  Forces  (Bundeswehr)  the  number  of  missions  in  foreign  countries  has  increased  in  the  last  two 
decades.  This  causes  increasing  demands  concerning  the  personality  of  soldiers.  These  new  challenges 
include  dealing  with  different  cultures  or  producing  adequate  behaviour  in  dangerous  situations. 
Concerning  the  answering  tendencies,  the  research  of  the  last  years  follows  two  different  directions.  First, 
the  answering  formats  are  varied  so  that  candidates  are  forced  to  decide  between  two  equivalent  answers 
to  suppress  faking.  In  this  case,  they  have  no  possibility  to  direct  their  answering  in  a  social  desirable 
manner.  This  approach  has  the  disadvantage  that  candidates  often  stop  answering  the  questionnaire.  The 
selection  between  two  equivalent  answers  often  means  a  choice  between  the  devil  and  the  deep  blue  sea. 
Second,  measurement  of  personality  can  be  implemented  in  more  or  less  real  life  situations,  even  with  the 
computers.  These  simulations  require  reacting  in  complex  situations  so  that  candidates  are  not  aware  of 
the  psychological  aspects  that  are  measured.  Because  of  this  unawareness  they  are  not  able  to  control  their 
behaviour  in  a  social  desirable  way. 
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The  present  study  is  engaged  in  the  context  of  this  last  research  direction.  The  aim  of  the  study  is  to 
develop  an  objective  and  efficient  diagnostic  instrument  that  can  be  used  in  personnel  selection  situations. 
The  study  was  conducted  by  Industrieanlagen-Betriebsgesellschaft  mbH  (IABG)  under  the  order  and 
supervision  of  the  Psychological  Department  of  the  German  Ministry  of  Defence.  IABG  used  an  improved 
version  of  the  so  called  assessment  demonstrator  and  the  methodology  of  Human  Behaviour 
Representation  (HBR)  to  develop  a  new  assessment  instrument.  As  a  last  step  of  the  study,  the  HBR- 
Monitor  shell  be  used  to  generate  an  automatic  report  of  the  diagnostic  results,  so  that  a  personnel 
selection  recommendation  can  be  made. 

1.1  The  HBR  Approach 

HBR  uses  methods  of  cognitive  modelling.  The  idea  of  the  approach  is  the  existence  of  a  level  of  basic 
cognitive  mechanisms  that  serve  as  a  fundament  for  personal  attributes  on  the  one  hand  and  differences 
between  candidates  on  these  attributes  on  the  other  hand.  In  the  framework  of  HBR,  personality  can  be 
modelled  as  an  interaction  of  parameters  of  an  information  processing  system.  HBR  respectively  cognitive 
architectures  can  be  understood  as  “relatively  complete  proposals  about  the  structure  of  human  cognition” 
(Anderson,  1993,  p.  3).  These  proposals  allow  using  a  consistent  theoretical  basis  for  cognitive 
information  processing.  In  doing  so,  HBR  does  not  look  at  single  cognitive  mechanisms  or  single 
personality  attributes,  but  postulates  systems  that  describe  information  processing,  goal  achievement  and 
activity  initiation  as  a  whole  (ACT-R:  Anderson,  1983;  SOAR:  Newell,  1992;  PSI:  Domer,  1999;  Cogent: 
Cooper,  2002). 

The  HBR  approach  is  build  on  the  PSI  theory  of  Dorner  (1999)  that  models  personality  in  a  framework  of 
cognitive  architecture  as  patterns  of  information  processing  (Schaub,  2001).  Personality  in  this  framework 
is  defined  as  persistent,  so  that  the  structures  and  parameters  of  the  system  are  stable.  The  theory  does  not 
only  model  cognitive  processes,  it  also  deals  with  integrated  dynamic  models  of  actions,  motivations  and 
emotions.  So  it  is  possible  to  describe  psychological  functions  and  structures  that  go  beyond  single 
personality  factors.  The  postulated  structures  and  processes  enable  the  cognitive  system  to  achieve  goals 
and  satisfy  directly  or  indirectly  motives.  Goals  are  defined  as  activated  motives  that  are  able  to  control 
actions.  In  this  framework,  goals  are  the  central  structure  of  the  PSI  system.  Aspects  of  the  theoretical 
model  include  different  processes  of  goal  generation,  goal  selection  and  goal  achievement.  Independent  of 
these  processes  are  some  general  parameters  that  define  rules  of  information  processing  and  control  the 
functioning  of  the  system.  The  most  important  of  these  parameters  are  “degree  of  resolution”,  “level  of 
activity”,  “flexibility”  and  “extemalisation”,  which  are  defined  as  follows: 

•  “degree  of  resolution”:  accuracy  of  information  processing 

•  “level  of  activity”:  speed  of  information  processing 

•  “flexibility”:  probability  of  interruption  of  the  current  goal  by  the  activation  of  a  concurrent  goal 

•  “extemalisation”:  probability  of  interruption  of  the  current  goal  by  external  stimuli 

Candidates  can  be  distinguished  first  with  regard  to  their  actual  values  on  these  parameters  and  second 
with  regard  to  the  range  of  possible  values  of  the  parameters.  The  actual  values  of  the  parameters  reflect 
the  adaptation  of  the  information  processing  system  to  the  situational  demands.  The  range  of  possible 
values  marks  the  upper  and  lower  boundary  of  the  capability  of  the  information  processing  system  to  adapt 
to  special  situational  demands.  With  respect  to  the  measurement  of  personality  this  means  that  one  has  to 
deal  with  both  aspects.  The  actual  parameter  value  shows  the  degree  to  which  a  person  has  adapted  to  a 
situation.  The  range  of  values  gives  a  deeper  insight  to  the  ability  of  a  candidate  to  adapt  to  different 
situations. 

The  HBR  approach  as  well  as  the  PSI  theory  has  been  tested  in  several  computer  simulations.  These 
simulations  have  compared  the  postulated  information  processing  system  with  the  behaviour  of  real-life 
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persons  (cf.  Dorner,  2000;  Schaub,  1993,  2001).  The  results  support  the  validity  of  the  HBR  approach 
respectively  the  PSI  theory. 

1.2  HBR  Simulations  -  The  “guard”  and  the  “stray”  Scenar 

The  present  study  employs  the  PSI  theory  respectively  the  HBR  approach  to  develop  two  different 
computer  based  simulations  (Schaub  &  Stroscher,  2008).  The  simulations  are  objective,  standardised  and 
application-oriented  diagnostic  instruments.  Their  goal  is  to  identify  desirable  and  important  personality 
attributes  for  applicants  and  /  or  occupants  of  military  positions.  The  simulations  are  called  scenars.  The 
first  (“guard”  scenar)  deals  with  the  interaction  of  a  simulated  guard  with  a  burglar  in  a  business  building. 
The  second  (“stray”  scenar)  relates  to  a  group  of  persons  visiting  a  town. 

The  scenars  represent  demands  that  can  be  found  in  either  military  or  civil  contexts.  The  candidates  are 
confronted  with  complex  and  risky  situations  in  the  framework  of  a  computer  simulation.  They  have  to 
make  critical  decisions  by  taking  over  the  role  of  the  protagonist  of  the  scenar.  By  steering  the  protagonist 
through  a  virtual  room  with  several  critical  situations,  the  decisions  can  be  recorded  and  evaluated  with 
regard  to  the  personal  attributes  in  focus.  The  framework  of  the  scenars  offer  the  possibility  to  bring  up  a 
cover  story,  in  which  the  candidates  are  engaged  without  realizing  which  aspects  of  their  behaviour  are 
monitored  and  what  kind  of  evaluation  is  done.  So  the  options  to  show  socially  desirable  behaviours  are 
minimal  and  personality  can  be  measured  “as  it  is”.  In  the  following  the  two  different  scenars  “guard”  and 
“stray”  are  described  more  closely. 

In  the  “guard”  scenar  the  guard  is  responsible  for  controlling  a  stock  depot.  The  depot  consists  of  several 
rooms  inside  a  building  and  an  area  outside.  At  the  beginning  the  guard  is  inside  the  guard-room.  Before 
starting  the  measurement  the  candidate  gets  detailed  information  on  how  to  steer  the  guard  inside  the 
scenar.  He  can  take  as  long  as  he  needs  to  exercise  the  steering  so  that  at  the  beginning  of  the 
measurement  section  the  influence  of  familiarity  with  the  necessary  computer  handling  capabilities  is 
minimal.  In  addition  to  that,  the  candidate  gets  some  framing  information  about  his  task  as  a  guard,  about 
the  building  in  which  he  finds  himself  and  about  some  additional  duties  in  the  course  of  the  action. 

After  starting  the  measurement  section  the  guard  hears  an  undefined  noise  from  outside.  From  now  on  he 
has  several  options  how  to  react  to  the  specific  situations  developing.  He  can  do  nothing,  he  can  go  and 
investigate  the  noise,  he  can  try  to  call  the  police  and  so  on.  Depending  on  how  he  reacts  the  scenar  is 
continued.  The  scenar  is  designed  in  such  a  way  that  the  guard  discovers  a  burglar  and  he  has  to  react  in  an 
adequate  manner.  Fulfilling  the  idea  of  a  simulation  many  options  lead  to  this  point.  Every  decision  of  the 
candidate  leads  to  different  outcomes,  which  are  recorded.  His  actions  on  his  way  to  capture  the  burglar 
define  different  aspects  of  personality  measurement. 

In  the  “stray”  scenar,  the  candidate  takes  over  the  roll  of  one  person  in  a  group  of  five.  The  situation  is 
defined  so  that  the  group  has  strayed  in  a  city.  At  the  beginning  of  the  scenar,  one  of  the  members  of  the 
candidates’  group  has  an  accident  so  that  he  needs  medical  help.  Several  tasks  are  defined  for  the 
candidate  in  the  scenar:  He  has  to  reach  the  railway  station,  the  group  has  to  reach  the  railway  station,  too, 
he  has  to  coordinate  his  actions  with  the  rest  of  the  group,  he  should  consider  the  danger  caused  by 
hooligans  and  he  should  keep  in  mind  that  no  one  in  the  group  is  armed.  In  the  course  of  action,  the  group 
meets  other  groups  (e.g.  hooligans)  or  single  persons  (someone  who  can  help  find  the  way  to  the  railroad 
station).  The  candidates  group  suggests  to  the  candidate  to  take  the  leading  role  by  asking  for  advice, 
acting  indecisive  and  so  on.  The  choice  of  actions  the  candidate  takes  over  the  course  of  the  scenar  offers 
ample  opportunities  to  measure  personal  abilities  especially  concerned  with  interaction  and  /  or  leading 
behaviour. 
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1.3  Construct  Model  for  HBR  Scenars 

For  both  the  “guard”  and  the  “stray”  scenar,  a  complex  and  consistent  construct  model  is  defined.  This 
model  is  described  in  Table  1 . 


Table  1 :  Description  of  constructs  measured  by  HBR-Tools. 


Construct 

Content 

Decision  finding 

Heuristic  competence 

(What  is  done?) 

Modulation 

(How  is  it  done?) 

Planning 

Searching,  identification  and  evaluation  of  options 
to  act 

Implementation 

Implement  options  to  act 

Intentions 

Conscious  selection  of  goals,  setting  priorities  for 
goals  and  subgoals 

Situation  awareness 

Analysis 

Information  gathering 

Mental  Model 

Anticipation  of  problems,  control  of  background 

Reflection 

Checking  and  evaluation  of  results 

Activity 

Active  handling  of  the  situation 

Rule-conformity 

Accounting  for  rules 

Risk-awareness 

Accounting  for  risks 

Sharedness 

Sharing  own  thoughts  with  others 

The  organisation  and  definition  of  constructs  is  based  on  the  HBR  approach.  Every  single  construct  exists 
in  two  independent  specifications.  There  is  a  positive  as  well  as  a  negative  specification  so  that  the  model 
as  a  whole  includes  20  different  factors.  The  idea  behind  the  independence  of  the  positive  and  negative 
aspects  of  the  same  constructs  is  that  the  information  processing  system  in  the  background  of  the  construct 
model  allows  showing  either  positive,  negative  or  both  specifications  depending  on  the  specific 
parameters  of  the  system.  For  example,  it  might  be  that  a  candidate  shows  good  rule -conformity  in  one 
moment  and  no  rule-conformity  in  the  next.  Following  the  HBR  approach  this  is  something  that  must  not 
be  averaged  out,  but  shows  two  different  aspects  of  the  information  processing  system.  Clearly  stated,  it 
shows  the  range  of  parameter  values  a  distinct  aspect  of  the  information  processing  system  may  take. 


2.0  METHOD 

The  following  section  first  describes  the  sample  used  in  this  study.  Second  it  gives  an  overview  of  the 
basic  structure  of  the  scenars  on  which  the  statistical  analysis  relies  upon. 

2.1  Sample 

Both  the  “guard”  scenar  and  the  “stray”  scenar  were  given  to  a  sample  of  applicants  respectively  members 
of  German  Special  Forces.  49  subjects  were  tested  with  the  “guard”  scenar,  55  subjects  completed  the 
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“stray”  scenar.  All  subjects  were  male,  neither  their  age  nor  their  educational  level  is  known.  The  “guard” 
scenar  was  given  to  a  sample  of  applicants  for  German  army  divers  in  addition.  These  1 1  subjects  were  all 
male,  too.  Similarly  to  the  first  sample  neither  their  age  nor  their  educational  level  is  known. 

2.2  Statistical  Analysis 

Statistical  analyses  for  both  scenars  were  based  on  so  called  behavioural  markers.  These  were  defined  as 
follows.  First,  both  scenars  were  divided  in  several  scenar  elements.  These  scenar  elements  include  in  the 
case  of  the  “guard”  scenar  the  different  rooms  of  the  stock  building  in  which  the  guard  can  act  as  well  as 
some  distinguishable  locations  at  the  outside.  In  the  case  of  the  “stray”  scenar,  scenar  elements  are  defined 
along  the  different  contexts  in  which  the  candidate  can  show  his  personal  abilities.  For  example,  there  is  a 
scenar  element  named  “meet  the  hooligans”  and  one  named  “person  who  can  help  find  the  way  to  the 
railway  station”. 

Within  these  scenar  elements  specific  behaviours  can  be  observed.  Concerning  the  “stray”  element,  a 
specific  behaviour  of  the  candidate  might  be  to  ask  the  members  of  his  group  for  ideas  to  get  to  the 
railway  station,  another  one  might  be  to  leave  the  group  after  one  of  the  group  members  has  had  the 
accident.  These  different  specific  behaviours  are  called  behavioural  markers. 

The  first  step  in  the  development  of  the  evaluation  rules  for  the  two  scenars  was  to  scan  all  different  scenar 
elements  for  possible  behavioural  markers.  In  this  first  step  there  was  no  guidance  with  respect  to  what  the 
behavioural  markers  might  measure.  Only  the  fact  that  a  decision  of  the  candidate  (in  the  sense  of  PSI 
theory;  Dorner,  1999)  might  result  in  a  specific  behaviour  and  the  expectation  that  candidates  might  react 
differently  in  their  respective  behaviours  is  the  rule  for  defining  an  action  as  a  behavioural  marker.  A 
differentiation  is  made  between  behavioural  markers  that  belong  to  exactly  one  scenar  element  and 
markers  that  occur  in  different  scenar  elements. 

In  a  second  step  the  behavioural  markers  were  arranged  with  respect  to  the  construct  model  shown  in  table 
1.  Besides  the  allocation  to  a  specific  construct  a  decision  had  to  be  made  if  the  behavioural  marker 
defines  either  the  positive  or  the  negative  aspect  of  the  construct. 

It  is  important  to  notice  that  both  the  definition  of  behavioural  markers  and  the  allocation  of  behavioural 
markers  to  a  specific  construct  are  preliminary.  Up  to  now,  the  theoretical  basis  for  measuring  personality 
in  an  objective  simulation  of  a  more  or  less  military  situation  is  small.  Rarely  information  can  be  given 
about  behavioural  aspects  that  are  able  to  reflect  personal  abilities.  So  this  study  cannot  rely  on  a  strong 
theoretical  foundation,  since  one  of  its  goals  is  to  produce  such  a  foundation  or  at  least  makes  one  step  in 
this  direction.  If  this  study  is  successful  it  can  serve  as  a  basis  for  future  research  with  a  more  fine-grained 
goal  structure. 

As  an  example  of  behavioural  markers  in  the  “guard”  scenar  that  belong  exactly  to  one  scenar  element,  the 
behavioural  markers  for  the  first  element  “guard-room”  are  named  here: 

•  Looks  around  in  the  room  (Constructs:  Activity  +,  Analysis  +) 

•  Takes  torch  (Constructs:  Planning  -,  Activity  +,  Mental  Model  +) 

•  Switches-on  torch  while  leaving  the  room  (Constructs:  Risk-awareness  +) 

•  Opens  door  <  2  minutes  (Constructs:  Activity  +,  Rule-conformity  +) 

•  Opens  door  with  switched-on  torch  (Constructs:  Risk-awareness  +,  Mental  Model  +) 

•  Tests  torch  (Constructs:  Analysis  +) 

•  Turns  the  radio  down  (Constructs:  Implementation  +,  Activity  +,  Mental  Model  +) 

•  Leaves  guard-room  (Constructs:  Implementation  +,  Activity  +,  Intentions  +,  Rule-conformity  +) 
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•  Opens  door  several  times  (Constructs:  Implementation  +) 

•  Torch  switched -off  while  leaving  the  guard-room  (Constructs:  Risk-awareness  -,  Mental  Model  -) 

The  alternative  of  defining  behavioural  markers  that  are  relevant  in  several  scenar  elements  can  be 
demonstrated  by  describing  some  of  the  markers  in  the  “guard”  scenar  (for  the  sake  of  simplicity  without 
naming  the  respective  constructs): 

•  Reads  instructions  first  time 

•  Reads  instructions  repeatedly  (beyond  three  times) 

•  Sounds  the  alarm 

•  Sounds  the  alarm  hastily 

•  Waits  for  the  police 

•  Protocols  controlling  of  interior  rooms  (after  investigating  them) 

•  Protocols  controlling  of  interior  rooms  (without  investigating  them) 

•  Protocols  controlling  of  outside  area 

•  Tiptoes  (>  10  sec) 

•  Walks  without  tiptoeing  (>  10  sec) 


By  splitting  both  scenars  in  several  scenar  elements  and  defining  behavioural  markers  for  every  scenar 
element  all  requirements  are  fulfilled  to  measure  personality  aspects. 

3.0  RESULTS 

The  following  section  shows  the  results  for  the  “guard”  and  the  “stray”  scenar.  Both  scenars  are  separately 
analyzed.  The  order  of  analysis  is  as  follows. 

First,  variation  of  the  reactions  with  respect  to  each  behavioural  marker  is  checked.  Since  this  is  a  first 
evaluation  of  the  simulations,  this  is  a  very  important  step.  Only  behavioural  markers  with  ample  variation 
can  reflect  real-life  differences  in  personal  abilities  between  persons.  In  classical  item  analysis  procedures, 
an  often  used  criterion  for  ample  variation  is  the  usage  of  item  means  between  M  =  .20  and  M  =  .80. 
Because  of  the  evaluative  nature  of  this  study  and  the  relatively  small  sample  size  this  criterion  is  widened 
and  all  behavioural  markers  are  kept  in  the  study  with  means  between  M  =  .  10  and  M  =  .90. 

Second,  the  postulated  construct  model  is  evaluated.  This  is  done  by  using  structural  equation  modelling 
procedures  (Joreskog,  1993).  This  is  done  despite  the  fact  that  the  sample  size  is  quite  small.  The  idea  is 
not  to  prove  that  the  construct  model  holds  with  all  facets  but  to  get  a  first  glance  to  what  extent  there  are 
deviations  from  the  model.  The  construct  model  can  be  analyzed  in  three  different  layers.  At  the  first  layer 
all  single  constructs  independent  of  their  positive  or  negative  values  are  considered.  So  at  this  first  layer 
there  are  20  different  models  to  analyze.  At  the  second  layer  all  behavioural  markers  developed  for  the 
positive  and  negative  aspects  of  identically  named  constructs  are  combined.  So  at  this  layer  there  are  10 
different  constructs.  At  a  third  layer  all  ten  constructs  are  united  and  analyzed  together,  so  there  is  only 
one  single  big  model  to  be  analyzed  at  this  layer.  The  models  at  the  three  layers  are  nested.  This  means 
that  only  if  a  successful  analysis  can  be  done  at  the  first  layer  it  is  necessary  to  go  on  to  the  second  layer. 
The  fit  of  the  model  at  higher  layers  cannot  be  better  than  the  fit  at  earlier  stages  of  the  models. 

Third,  classical  explanatory  factor  analysis  is  done  to  find  out  what  potential  alternative  structure  can  be 
identified  in  the  data.  These  analyses  can  be  the  base  for  further  developments  of  the  scenars.  Explanatory 
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factor  analysis  is  done  by  main  component  analysis  using  varimax  rotation.  Criterion  for  extraction  of 
models  is  the  scree  plot. 


Table  2:  Results  of  SEM  -  “Guard”  scenar 1). 


Nr. 

Factor 

Number  of 
Indicators 

Chi-Square 

df 

P 

GFI 

CFI 

RMSEA 

1 

Planning  (positive) 

5 

- 

- 

- 

- 

- 

- 

2 

Planning  (negative) 

2 

- 

- 

- 

- 

- 

- 

3 

Implementation  (positive) 

9 

35.116 

27 

.136 

.883 

.709 

.071 

4 

Implementation  (negative) 

2 

- 

- 

- 

- 

- 

- 

5 

Intentions  (positive) 

7 

- 

- 

- 

- 

- 

- 

6 

Intentions  (negative) 

3 

- 

- 

- 

- 

- 

- 

7 

Analysis  (positive) 

10 

- 

- 

- 

- 

- 

- 

8 

Analysis  (negative) 

3 

- 

- 

- 

- 

- 

- 

9 

Mental  Model  (positive) 

12 

- 

- 

- 

- 

- 

- 

10 

Mental  Model  (negative) 

11 

- 

- 

- 

- 

- 

- 

11 

Reflexion  (positive) 

1 

- 

- 

- 

- 

- 

- 

12 

Reflexion  (negative) 

2 

- 

- 

- 

- 

- 

- 

13 

Activity  (positive) 

10 

63.051 

35 

.003 

.846 

.647 

.117 

14 

Activity  (negative) 

6 

- 

- 

- 

- 

- 

- 

15 

Rule -conformity  (positive) 

8 

- 

- 

- 

- 

- 

- 

16 

Rule -conformity  (negative) 

7 

19.977 

14 

.131 

.914 

.821 

.085 

17 

Risk-awareness  (positive) 

11 

- 

- 

- 

- 

- 

- 

18 

Risk-awareness  (negative) 

8 

- 

- 

- 

- 

- 

- 

')  in  the  columns  for  the  fit  indices  indicate  that  the  respective  model  couldn’t  be  estimated. 
Explanations  are  given  in  the  text. 

3.1  The  “guard”  Scenar 

Concerning  the  variation  of  the  answers  43  of  the  77  behavioural  markers  measured  reach  the  criterion  of 
mean  between  M  =  .10  and  M  =  .90.  Ten  of  them  are  measured  in  several  scenar  elements,  eight  belong  to 
scenar  element  I,  three  belong  to  scenar  element  II,  six  belong  to  scenar  element  III,  one  belongs  to  scenar 
element  IV,  two  belong  to  scenar  element  V,  eight  belong  to  scenar  element  VI  and  five  belong  to  scenar 
element  VII.  Concerning  the  distribution  of  behavioural  markers  across  the  different  scenar  elements,  a 
noticeable  reduction  of  behavioural  markers  in  scenar  elements  II,  IV  and  V  must  be  mentioned.  In  order 
to  construct  an  effective  and  efficient  measurement  instrument  this  should  be  corrected  in  later  versions  of 
HBR-Tools. 

The  postulated  construct  model  is  evaluated  using  the  43  behavioural  markers  that  remain  after  analyzing 
the  variation  of  the  behavioural  markers.  The  fit  of  the  structural  equation  models  (SEM)  used  is 
investigated  through  Chi-square,  GFI,  CFI  and  RMSEA.  For  the  guard  scenar  18  of  the  possible  20 
constructs  at  the  first  level  are  relevant.  The  construct  “Sharedness”  (positive  and  negative)  is  not 
measured  because  of  its  interactive  character,  while  the  guard  has  to  walk  alone  through  the  scenar.  Table 
2  shows  the  results  of  SEM. 
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Only  three  of  the  18  models  could  be  estimated.  Three  possible  reasons  can  be  given  for  that.  First,  the 
sample  size  is  small  for  using  SEM.  This  might  lead  to  estimation  problems.  Second,  the  reduction  of 
behavioural  markers  through  variation  analysis  leads  for  a  couple  of  scales  to  a  number  of  indices  lower 
than  four.  For  these  scales  a  stable  estimation  is  problematic.  Third,  some  of  the  behavioural  markers 
belonging  to  one  scale  correlate  r  =  1.0.  Models  with  this  characteristic  cannot  be  estimated.  The  second 
and  third  reason  call  the  postulated  construct  model  into  question.  Because  these  two  reasons  are  valid  for 
all  15  models  that  are  not  estimated  and  due  to  the  fact  that  the  remaining  three  models  don’t  have 
satisfactory  fit  indices  the  postulated  model  must  be  questioned.  Despite  the  preliminary  character  of  the 
analysis  made  here  the  character  of  the  results  indicate  that  the  construct  model  must  have  a  substantially 
different  character. 

Hence  the  43  behavioural  markers  remaining  after  variation  analysis  are  used  in  explanatory  factor 
analysis.  The  scree  plot  indicates  six  factors.  These  factors  can  be  given  nameable  content  after  varimax- 
rotation.  The  meaning  of  the  factors  and  the  allocation  of  behavioural  markers  to  the  factors  are  shown  in 
table  3. 


Table  3:  Explanatory  factor  analysis  -  “Guard”  scenar. 


Factor 

Naming 

Behavioural  Markers 

1 

Risk-aware  investigation 
behaviour  -  General 
aspects 

-  gun  not  hold  ready 

-  gun  hold  ready 

-  risk-aware  investigation 

-  thoroughness 

-  goes  directly  to  the  outside  area 

-  door  of  the  sensible  room 

-  protocol  “indoor  rooms”  ok 

2 

Preparing  for 
investigations 

-  opens  door  <  2  min 

-  tests  torch 

-  opens  door  several  times 

-  torch  switched-off  while  leaving  the  room 

-  torch  switched-on  while  leaving  the  room 

-  takes  torch 

-  goes  into  the  guard-room 

-  opens  door  while  torch  is  switched-on 

-  turns  down  radio  (<20%) 

3 

Risk-aware  investigation 
behaviour  -  Concrete 
aspects 

-  closes  the  door 

-  does  not  tiptoe  (10  sec) 

-  switches-on  light 

-  door  of  sensible  room  not  tested 

-  waits  >20  sec 

-  overhasty  alarm 

-  switches-off  light 

-  door  to  outside  area  not  closed 

-  gun  hold  ready 
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Table  3:  Explanatory  factor  analysis  -  “Guard”  scenar,  continued. 


Factor 

Naming 

Behavioural  Markers 

4 

Consideration  of  general 
framework 

-  reads  instructions  first  time 

-  doesn’t  read  instructions 

-  protocol  „indoor  rooms"  not  ok 

-  tiptoes  (10  sec) 

-  opens  door  quietly 

-  opens  protocol 

-  looks  around  in  the  room 

-  doesn’t  see  hole  in  the  fence 

5 

Handling  of  concrete 
danger 

-  gun  not  hold  ready  while  seeing  the  burglar 

-  doesn’t  pull  gun 

-  sees  burglar 

-  notices  window 

-  asks:  Who  are  you? 

-  walks  away 

6 

Handling  of  concrete 
danger 

-  pulls  gun 

-  pushes  alarm  button 

-  alarm  button 

-  says:  hands  up 

-  torch  stays  switched-on 

-  doesn’t  follow  burglar 

-  speaks  while  gun  is  pulled 

-  door  of  sensible  room 

Following  the  results  of  explanatory  factor  analysis,  the  “guard”  scenar  offers  the  possibility  of  measuring 
different  aspects  of  risk-aware  investigation  behaviour.  The  six  factors  identified  can  be  allocated  to  five 
different  aspects  with  regard  to  content. 

3.2  The  “stray”  Scenar 

Only  17  of  the  51  behavioural  markers  show  ample  variation.  Concerning  the  postulated  construct  model 
this  reduction  leads  to  the  problem  that  for  a  large  number  of  the  constructs  the  number  of  behavioural 
markers  is  too  small.  For  that  reason  the  inspection  of  the  postulated  construct  model  is  not  carried  out. 

Although  one  of  the  tasks  for  further  development  of  the  “stray”  scenar  has  to  be  the  increase  of  the 
number  of  behavioural  markers,  the  residual  markers  after  the  variation  analysis  were  investigated  through 
explanatory  factor  analysis.  Only  one  factor  can  be  extracted.  In  addition,  this  factor  represents  only 
eleven  of  the  17  behavioural  markers  remaining  after  variation  analysis.  The  results  are  shown  in  table  4. 
Despite  the  fact  that  content  can  be  found  in  the  “stray”  scenar,  this  content  is  based  only  on  a  minimal 
number  of  behavioural  markers  which  calls  the  functioning  of  the  scenar  into  question. 

At  the  current  stage  of  development  the  “stray”  scenar  can  be  seen  as  measuring  communication  and 
teamwork  abilities.  The  postulated  construct  model  could  not  be  found  at  this  time. 
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Table  4:  Explanatory  factor  analysis  -  “Stray”  scenar. 


Factor 

Naming 

Behavioural  Markers 

1 

Communication  and 
teamwork 

-  Says:  Can  you  show  us  the  way  to  the  railway  station? 

-  Says:  Thank  you  for  the  location  map. 

-  Says:  My  friends  do  have  a  location  map.  Let’s  visit  them 

-  Says:  Please  show  the  pedestrian  the  location  map,  so  he  can  show 
us  the  way. 

-  Candidate  doesn’t  speak  to  the  pedestrian 

-  Says:  Sorry,  1  don’t  have  a  location  map.  Good  bye! 

-  Candidate  is  not  near  his  group 

-  Candidate  follows  the  group 

-  Candidate  catches  up  to  the  group 

-  Says:  Please  calm  down.  We  don’t  want  any  trouble. 

-  Takes  flight 

4.0  DISCUSSION 

For  both  scenars,  different  conclusions  must  be  drawn  out  of  the  analysis. 

4.1  The  “guard”  Scenar 

First,  a  revision  of  the  scenar  seems  to  be  necessary.  The  number  of  behavioural  markers  is  substantially 
reduced  through  variability  analysis.  This  means  reduction  of  efficiency  and  effectiveness.  In  addition, 
with  respect  to  the  measurement  model  the  number  of  behavioural  markers  is  not  big  enough.  The  revision 
is  necessary,  too,  because  a  substantial  number  of  behavioural  markers  show  very  high  correlations.  As  a 
result  of  a  revision,  a  finer  measurement  model  with  optimally  allocated  behavioural  markers  must  be 
developed. 

The  confirmatory  and  exploratory  factor  analyses  have  shown  that  the  postulated  construct  model  could 
not  be  found  in  the  data.  The  alternative  results  of  exploratory  factor  analysis  offer  an  interesting 
alternative  construct  model.  Risk-aware  behaviour  becomes  more  and  more  important  the  higher  the 
stakes  will  be  in  military  applications.  So  a  differential  measurement  instrument  will  be  necessary  for 
several  applications.  The  recommendation  after  the  analysis  of  these  first  data  described  here  should  be  to 
further  develop  the  measurement  model  of  “risk-aware  investigation  behaviour”  foreshadowed  here. 

4.2  The  “stray”  Scenar 

The  reduction  of  behavioural  markers  after  the  variability  analysis  is  more  distinct  in  the  “stray”  scenar 
than  in  the  “guard”  scenar.  After  the  variation  analysis  only  17  indicators  remain,  wherefore  the  further 
SEM  respectively  explanatory  factor  analyses  are  based  on  a  small  basis.  This  leads  to  the  question  of 
efficiency  and  effectiveness  of  the  “stray”  scenar.  Because  of  the  high  costs  of  development  for  the  scenar, 
the  number  of  measurement  indicators  isn’t  great  enough.  As  far  as  the  “stray”  scenar  is  concerned,  further 
recommendations  for  content  development  should  wait  until  the  number  of  suitable  behavioural  markers  is 
substantially  increased. 
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